feat: add RAGAS adapter for third-party eval metrics - #618

Summary

Adds RAGASAdapter to the third-party evaluation adapters, alongside the existing DeepEvalAdapter and AutoEvalsAdapter. Wraps any RAGAS metric as an AgentCore code-based Lambda evaluator with the same BaseAdapter pattern.

Design

Scores metrics through RAGAS's per-sample APIs (metric.single_turn_score() / metric.multi_turn_score() for legacy metrics, metric.score(**kwargs) for ragas.metrics.collections metrics) rather than the batch ragas.evaluate() pipeline. This means the adapter itself adds no dependency on datasets/pyarrow/pandas, which makes it compatible with slim ragas builds for size-constrained Lambda deployments.

Three-way dispatch:

• Legacy single-turn (Faithfulness, ExactMatch, ContextPrecision, ...): SingleTurnSample + single_turn_score()
• Legacy multi-turn (ToolCallAccuracy, TopicAdherenceScore, AgentGoalAccuracy): MultiTurnSample + multi_turn_score()
• Collections / decorator metrics (ragas.metrics.collections, @discrete_metric, @numeric_metric): metric.score(**fields) with signature-based field filtering

Features

• Validates metric.required_columns before scoring — returns MISSING_REQUIRED_FIELD with actionable guidance instead of letting ragas return silent 0.0 scores
• Parses ground truth and retrieval context embedded in the user message (\n\nReference Answer:\n and \n\nContext:\n separators) since ADOT trace formats have no dedicated fields for these
• Recovers ranked chunk lists from JSON-serialized embedded contexts, preserving chunk boundaries for rank-aware metrics (LLMContextPrecision)
• Handles threshold=None metrics (e.g. SemanticSimilarity) and provides an adapter-level threshold= override for collections metrics which carry none
• Discrete (string-valued) metrics produce categorical labels natively
• Preserves MetricResult.reason as the explanation
• ImportError at score time returns MISSING_DEPENDENCY (not a generic crash)
• Multi-turn messages are stripped of embedded markers so ground truth doesn't leak into the conversation being judged
• RAGASAdapter is the canonical class; RagasAdapter alias provided

Usage

from bedrock_agentcore.evaluation.custom_code_based_evaluators import custom_code_based_evaluator
from bedrock_agentcore.evaluation.custom_code_based_evaluators.third_party.ragas import RAGASAdapter
from ragas.metrics import Faithfulness
from ragas.llms import LangchainLLMWrapper
from langchain_aws import ChatBedrockConverse

eval_llm = LangchainLLMWrapper(ChatBedrockConverse(
    model_id="us.anthropic.claude-sonnet-4-5-20250929-v1:0",
    region_name="us-east-1",
))
adapter = RAGASAdapter(metric=Faithfulness(), llm=eval_llm)

@custom_code_based_evaluator()
def handler(eval_input, context=None):
    return adapter(eval_input, context)

Known Limitations

• reference_contexts (ground-truth contexts for reference-comparison metrics) is not defaulted from retrieved contexts — supply it via custom_mapper when a genuine ground-truth source exists
• reference_tool_calls built from expected_trajectory.toolNames carry names without arguments; arg-accurate comparison needs a custom_mapper with full ToolCall objects
• reference_topics (TopicAdherenceScore) has no span source — requires custom_mapper
• Dual-mode metrics (AspectCritic, SimpleCriteriaScore) always score single-turn
• ragas <1.0 imports datasets when the ragas package is imported; the adapter is compatible with trimmed builds but does not solve that import itself

Dependency Notes

• Pins ragas>=0.4.3,<1.0.0 — the tested version range
• Pins langchain-community>=0.3.0,<0.4.2 — langchain-community 0.4.2 removed chat_models.vertexai, which ragas <1.0 imports unconditionally
• Adds ragas to the evaluation integ test extra-deps in CI
• uv.lock regenerated for the new extra; incidental bumps to langchain-community resolution only

Testing

• 48 unit tests covering all dispatch branches, embedded parsing (plain text + JSON chunk lists), validation, multi-turn messages, threshold override, error codes
• 8 integration tests with real deterministic metrics (legacy ExactMatch, collections ExactMatch, @discrete_metric, ToolCallAccuracy match/mismatch, adapter-imports-without-datasets)
• All pre-commit hooks pass; uv lock --check passes